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3D GRATING OPTICAL SENSOR HAVING A DIFFUSION 
GLASS FOR CONDUCTING CHROMATOMETRY WITH 
COLOR CONSTANCY PERFORMANCE 

BACKGROUND OF THE INVENTION 

[0001] A grating optical sensor is disclosed in WO 97/22 849. It is 
provided for accurately determining spatial and/or temporal spacings in 
focused image sequences of a lens/pupil system and/or determining 
spatial and/or temporal object parameters in real time such as, for 
example, speed or depth. A 3D grating has also already been used to 
carry out model calculations relating to the inverted retina of the human 
eye and to relate them to subjective phenomena known from human 
vision. In the preferred form, the 3D grating has a hexagonal structure. 
Other structures with centrosymmetrical diffraction patterns are, 
however, likewise possible. 

[0002] Since the investigations of O. Lummer and the industrial 
development of daylight-like luminaires, it has been realized that there 
is an as yet unexplained resonance between sunlight and human vision. 
This has resulted in all the previous recommendations for approximating 
the spectrum of artificial light sources to the sunlight spectrum. In 
particular, in the case of color perception in phototopic day vision, 
there occur in the event of a change of illuminations having a different 
spectral composition of the radiation displacements of the color values 
which are compensated adaptively in human vision after a relatively 
short or, in part, relatively long time by means of approximate color 
constancy performances of the eye. The v. Kreis model, which 
attributes the adaptivity to the visual pigments of the retina, presently 
serves as an incomplete explanatory model for this. In addition, there 
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are even more incomplete cortical explanatory models from other 
authors. 

[0003] On the other hand, it has been documented many times that 
the phototopic seeing process cannot be characterized solely by the 
spectral light sensitivity of the individual cones. The very much more 
complex mode of operation of the visual sense requires knowledge of 
the luminance distribution in the entire visual field for the purpose of 
judging many visual tasks. Human vision is not based on the stimulus/ 
reaction response of individual pixels. It takes account of the relative 
values over the entire field of view. In addition to chromatic adaptation 
effects, scattering of light at ocular media influences the extent of the 
achromatic axis (black-gray-white axis) centering the color space. It is 
therefore an illusion to believe that spectral photometers will be the 
ideal instruments of future chromatometry and color determinations, 
even if they are designed on the detection of overlapping RGB values. 
Likewise incomplete is a chromatometric technique which respectively 
dispenses with determining the triad of brightness/hue/saturation 
simultaneously and with reference to a entire field of view. 

SUMMARY OF THE INVENTION 

[0004] There is thus a growing need to have available in the future 
color sensors which can measure color values with reference to the 
spectral sensitivity curves of human vision, and ensure, given 
adaptation to artificial illuminations, an approximate color constancy 
corresponding to human vision. It is the object of the invention to 
create such a sensor. 

[0005] The invention proceeds from the finding that it is possible, by 
inserting a diffractive multilayer (3D) grating into the image plane of an 
imaging lens/pupil system in the near field downstream of the grating 
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(Fresnel/Talbot space; Fourier space or reciprocal grating), to make 
available three chromatic diffraction orders (RGB triple) with in each 
case six discrete interference maxima on mutually concentric circles, 
such as are described in the case of a hexagonal grating structure by 
means of the v. Laue equation known from crystal optics. 
[0006] The v. Laue equation for diffractive space lattices requires for 
the production of constructive interference maxima the simultaneous 
satisfaction of the three phase conditions in the equation 1 1 | 
[0007] (cosa-cosa°) = hi X/gx 

[0008] (cosp-cosp°) = h2X/gy |1| 
[0009] (cosy-cosy °) = h3 X/gz 

[0010] (h1h2h3 = triple of integral diffraction orders n; a°, P°,y° = 
aperture angle of the light cone incident in the 3D grating, relative to x, 
y, z; a, p, y = angle of the diffraction orders relative to x, y, z; X = 
wavelength; and gx, gy, gz = grating constant in the x-, y-, z-axial 
direction). Assuming a hexagonal packing of the optically diffracting 
elements and grating constant dimensions in //m of gx = 2X1 1 1 , gy = 
4M 1 1/V3, gz = 4A,1 1 1, in equation |2|, M 1 1 constitutes the 
wavelength diffracted with maximum transmission into the 111 
diffraction order. 

~,hl o hi h3 

2( — coscz + — cosp + — cos^ ) 

[0011] ?ih1h2h3 = M11 = — — &— 2 ^ z \2\ 

-H -H J 

gx gy gz 

[0012] In the case of perpendicular incidence of light (a°=p°=90° / 
y = 0°) a triple of chromatic diffraction orders results in the visible 
spectral region (380-780 nm) 
[0013] M 1 1 (longest wavelength) RED 
[0014] M23 (average wavelength) GREEN 
[0015] XM2 (shorter wavelength) BLUE 
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[0016] The spectral transmission curves, which are centered relative 
to one of these A,max in each case, have a Gaussian shape and are 
determined at their half width by the number of the surface gratings in 
the z direction that are present in the 3D grating. In the event of 
incidence of white light, that is to say light of identical energy in all 
spectral components, and the grating inserted into the image plane of 
the imaging system, given the selection of M 1 1 = 559 nm, the result 
is the trichromatism of the diffraction orders at 
[0017] M 1 1 RED = 559 nm 
[0018] M23 GREEN = 537 nm 
[0019] M 22 BLUE = 447 nm 

[0020] There is thus a trichromatic tuning of the 3D grating, which is 
based on the resonant setting of the grating constants gx and gz to an 
integral A,1 1 1 , and in which a trichromatic equilibrium of the brightness 
values (Patterson amplitudes 2 weights) is produced in the RGB 
diffraction orders. 

[0021] In the case of adaptive chromatic retuning of the 3D grating 
to an illumination other than a white one, the relation of the RGB Xmax 
(1 : 0.96 : 0.8 or 25 : 24 : 20) is always maintained. A,1 1 1 is the 
resonant wavelength determining the triple shifts to shorter X1 1 1 
wavelengths in the event of change to a blue illumination, and to longer 
X*\ 1 1 wavelengths in the event of change to a red illumination. The 
adaptive shift ends with the complete adaptation to the new 
illumination, that is to say with the resonant finding of a new RGB 
equilibrium, of the trichromatically additive white standard, which 
recenters the color space. The actual resonance factor is the phase 
velocity nvA, = c (n = refractive index of the medium, v = frequency 
of the light, c = speed of the light). 
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[0022] The following new configuration of the sensor design forms 
the basis for the color constancy performance of the 3D grating optical 
sensor in the case of adaptation to variable illuminations. 
[0023] A diffusion glass or one or more light diffusing gratings are 
incorporated into the pupil plane (aperture space) of the imaging optical 
system. Their fiihction is to be seen in that they scatter diffusely as 
incoherent background into the image plane information likewise 
present at each location in the pupil, spatially the sum of the spectral 
intensities and loca\fte^jency values which are irradiated into the pupil 
by all objects in the pbjepr space and contribute to optical imaging. As 
a result, each locaHfraage location is supported by the global 
information on the entire field of view, against which each local pixel 
must stand out by beimg differentiated from it, specifically in 
brightness, hue, saturation etc. However, each item of local 
information thereby remains relativized in terms of the global 
background of the entire field of view. 

[0024] All the lenses, diffusion glasses or gratings are designed such 
that they are transparent exclusively for electromagnetic radiation in 
the visible spectrum (380-780 nm) and therefore these delimit an 
octave of the wavelengths or frequencies with definite absorption 
edges. This boundary condition is important because thereby spectral 
brightness values which could come about through variation in the 
illumination are cut off at these absorption edges. 

[0025] In the near field downstream of the diffractive 3D grating, the 
RGB interference maxima (3x6 concentric maxima) assigned to a local 
pixel are interconnected via photoreceivers set in a constant fashion in 
terms of their spectral sensitivity to white sunlight (of identical energy 
in all spectral components) in such a way that a local RGB sum can be 
formed as a trichromatically additive brightness value by means of an 

appropriate evaluation. It is possible thereby to differentiate RGB 
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equilibriums and disequilibriums. RGB equilibriums correspond in the 
object space to visible colorless surfaces or illuminations (black-gray- 
white objects). If illuminations are not visible, but can be inferred only 
via colorless objects or surfaces, they are represented in terms of their 
spectral characteristic by gray or white surfaces, what are termed 
mirrors of the illuminations. The image location with RGB equilibrium, 
which achieves the greatest aggregate brightness, supplies what is 
termed the white standard, and thereby defines the tip of the 
achromatic axis centering the color space. Alternatively, the image 
location whose RGB values most closely approximate an equilibrium 
takes over this provision of a white standard. This explains that the 
white standard can be displaced in the trichromatic space. 
[0026] The design of a diffractive 3D grating optical sensor which 
provides trichromatic RGB values in three diffraction orders ensures 
color constancy when there is ensured together with the sudden or 
gradual change in the illumination in the object space a resonant 
mechanism, that is to say one that is adaptive to the spectral 
composition of the illuminations in the 3D grating, which corresponds 
to a chromatic tuning of the 3D grating. In the case of a white 
illumination, that is to say an equal-energy spectral composition, 
corresponding to average sunlight, of the visible light, three grating 
constants in the xyz-axial direction are tuned to the RED wavelength 
(559 nm), with standing wave formation in the x- and z-axial 
directions, that is to say resonance in the 3D grating. Identical values, 
that is to say RGB equilibriums, result thereby under the three Gaussian 
spectral photopic curves of the photoreceptors (cones in human vision). 
The white standard is determined via the RGB sum values of the three 
Gaussian curves, which are centered relative to the wavelengths 559 
nm RED/537 nm GREEN/447 nm BLUE. 

-6- 
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[0027] After a sudden or gradual change in the illumination, a 
chromatically triggered reconstruction of the grating constants takes 
place in the diffractive 3D grating. In the case of a displacement of 
illumination to the longer wavelength region of the spectrum, the white 
standard in the 3D grating, still tuned to 559 nm RED, suddenly breaks 
down. If the adaptive mechanism of the shifting of the white standard 
then acts in the direction of the changed illumination, the grating 
achieves a new RGB equilibrium in the case of a chromatic tuning to 
728 nm RED, for example. The trichromatically additive color space is 
thereby centered again relative to an achromatic axis, and the colors 
are correct again, being experienced as correct. 
[0028] If, by contrast, the illumination is displaced toward the 
shorter wavelength spectrum, the grating achieves a new RGB 
equilibrium, for example in the case of a chromatic tuning to 51 3 nm 
RED. 

[0029] The adaptive process, which leads as a result to a 

trichromatic restandardization of the color space in a changed white 

standard, can be described by the already explained v. Laue equation of 

crystal optics. The actual resonance factor is the phase velocity in the 

medium. Spectral triggering of the grating constant dimensions 

corresponds to the coefficient of thermal expansion for the RED 

wavelength in the RGB triple. It is possible by means of dosed IR, that 

is to say thermal irradiation in the 3D grating, or by varying the internal 

pressure in the 3D grating to vary the grating constant dimensions 

correspondingly. The sensor according to the invention can thereby 

ensure the color constancy properties of the human vision system. 

[0030] A color constancy sensor which is represented technically in 

the form of a 3D grating or preprocessing filter that can resonate with 

the centroid of the spectral component of a light source or illumination 

is very important for all applications in which the color of fabrics and 
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materials must be detected, differentiated and classified by the color 
perception forming the basis of the laws of human vision. This also 
holds for the relevant judgment of properties of visible objects that are 
associated with the hue characteristics, whether this be in general 
image processing, whether with automatic viewers in robotics or 
autonomously driving vehicles, indeed even in the case of sensors for 
the blind. At the same time, such a sensor is able to render color 
perceptions under artificial light sources predictable and measurable. 
Since such a 3D grating transforms the physical parameters (intensity 
and wavelength) into the psychological triad of brightness, hue and 
saturation, it is also possible to use a sensor to calculate brightness 
and saturation values of object surfaces. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0031] An exemplary embodiment of the grating optical sensor 

according to the invention is illustrated diagrammatically in the drawing 

and described with the aid of the figures, in which: 

[0032] Fig. 1 shows the design of the sensor, 

[0033] Fig. 1a shows the plan view of a centrosymmetrically 

[0034] Fig. 2 shows the adaptation to a white illumination, 

[0035] Fig. 3 sh oyvs the adaptation to a red illumination, 

[0036] Fig. 4 shows the cycle of the operation of adapting to a red 

illumination, 

[0037] Fig. 5 shows the adaptation to a blue illumination, and 
[0038] Fig. 6 shows the cycle of the operation of adapting to a blue 
illumination. 
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DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

[0039] The grating optical sensor illustrated schematically in Fig. 1 
includes as lens/pupil system an imaging lens 1 . The latter projects the 
image of a visible object 2, which is illuminated by a radiation source 3 
emitting white light, from the object space onto a hexagonal diffractive 
3D grating optical modulator 4 with the grating constants gx, gy, gz in 
the image plane 5. Three chromatic RGB diffraction orders in the 
diffraction pattern 6 with six concentric maxima (Patterson weights) 
each result in a known way for each imaged object 2 in the visible 
spectrum due to diffraction in the modulator 4, resonance between 
A,1 1 1 and the grating constants, and interference in the near field 
downstream of the modulator 4. These are illustrated once again in 
plan view in Fig. 1a for an object 2 situated on the optical axis 7 of the 
sensor. In this case, the red (R) diffraction order is situated on the inner 
ring, the blue (B) on the middle ring, and the green (G) on the outer 
ring. 

[0040] Each diffraction order is assigned a photoelectric receiver 8. 
All the receivers 8 are set to the same spectral sensitivity for a 
radiation source 3 emitting white light. 

[0041] Objects situated outside the optical axis 7 supply identical 
diffraction patterns 6, which can also be interleaved. The resolution of 
the image depends on the grating constants of the modulator 4. Each 
diffraction pattern is assigned a specific object. 

[0042] A diffusion glass 9 is inserted into the pupil plane of the lens 
1 or a plane conjugate thereto. This diffusion glass 9 can 
advantageously have a diffracting grating structure. Since imaging 
beams from all the objects in the object space run through each 
location of the pupil, image information from the entire object space is 
distributed simultaneously over the image plane via each diffusion 
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center outward from the pupil plane. Consequently, information on the 
overall image is superimposed on each local image of an object. The 
diffusion characteristic of the diffusion glass 9 is to be selected such 
that diffusion takes place as uniformly as possible over the entire image 
field, and an image of the local object on the background produced by 
diffusion is maintained. 

[0043] The spectral transmission of the lens 1 , the diffusion glass 9, 
and the modulator 4 are limited to the visible region of electromagnetic 
radiation, in particular to the wavelength region of 380-780 nm. 
[0044] All the receivers 8, assigned to the same diffraction order R, 
G, B, of a diffraction pattern 6 are interconnected for the purpose of 
forming a local chromatically additive brightness value 10. The local 
trichromatically additive brightness values 1 1 are additionally produced 
therefrom downstream. 

[0045] The local chromatically additive and trichromatically additive 

brightness values 10, 11 of the diffraction pattern 6, and the 

corresponding brightness values 10\ 1 1 1 of other diffraction patterns 

are fed to a comparison arrangement 1 2 for the purpose of determining 

the diffraction pattern with the best agreement between the 

chromatically additive brightness values 10, 10' and, simultaneously, a 

maximum trichromatically additive brightness value 11, 1 1 '. The 

corresponding brightness values of the selected diffraction pattern are 

led to a white standard forming unit 13 for the purpose of producing a 

white standard signal or value. The agreement between the three 

chromatically additive brightness values means that a colorless object 

detail is involved. The magnitude of the trichromatically additive 

brightness value specifies an evaluation on the black-gray-white scale. 

[0046] The chromatically additive brightness values 10, 10* of the 

individual diffraction patterns can also be fed to a color value forming 

unit 14. The sum of the three different chromatically additive 
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brightness values that are referred in each case to the white standard 
signal, for example multiplied by the reciprocal of the white standard, 
forms the output signal for the local color value. If there is no measured 
white standard value available, the color evaluation can also be carried 
out by forming the ratio with a fictional white standard value. 
[0047] The process of white standard formation can be checked 
permanently or in periodic sequence in a variation signaling unit 15. A 
variation in the illumination leads either only to a change in the 
trichromatically additive brightness value 1 1 , 1 1 ' of the diffraction 
patterns selected for the white standard, while an RGB equilibrium is 
maintained. The white standard value is displaced in this case only on 
the achromatic axis, centering the color space, for black-gray-white 
objects. 

[0048] If the variation in the illumination also leads, however, to an 
RGB disequilibrium in the diffraction pattern determining the white 
standard, the cause resides in a variation in the spectral composition of 
the illumination. The variation signaling unit 15 establishes such a 
variation and controls an adapter 16 which is assigned a thermal 
radiation source 17. Thermal radiation on the modulator 4 causes 
variation in its grating constant as a function of the coefficient of 
thermal expansion until the white standard forming unit 13 displays a 
new white standard value. This adaptation process corresponds to an 
inclination of the achromatic axi s centering the color space. 
[0049] Fig. 2 firstly illustrates the adaptation to a white illumination 
with spectral components of approximately the same energy. The 
intensity of emission is illustrated in the upper diagram as a function of 
wavelength. A diffractive hexagonal 3D grating in this case supplies 
three diffraction orders whose Gaussian spectral transmission curves 
are centered in relation to X1 1 1 = x 559 nm (RED), XI 23 = 537 nm 
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(GREEN) and X122 = 447 nrt^(BLUE). This corresponds to the 
sensitivity of the cones in human day vision. The Gaussian curves 
illustrated in the lower diagramlcan be described by a" 1 exp(-x 2 ), with 
x = (Xh1 h2h3 - X)/n and a = 0592 at n = 55 for 1 1 1 RED, a = 0.88 at 
n = 53 for 1 23GREEN, and a J 0.56 at n = 34 for 1 22BLUE. The 
achromatic, that is to say gray to white, objects reproduce the spectral 
properties of the illumination iq the object space to the extent that this 
itself is invisible. The product df the spectral intensities and spectral 
Gaussian curves produce identi^a|brightness aggregate values of 33% 
each in the three diffraction orders. Their RGB equilibrium supplies the 
white standard, which centers meutrichromiatic color space. In the 
table below, the values of the spe|tral brightness distribution are 
summarized in accordance with the Gaussian curves assigned to the 
diffraction orders, for the case of alwhite illumination. 
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Spectral brightness values 
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447 nm 
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1 57 
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680 

\ 
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26 

29 
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1 

13 

14 
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6 

6 
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2 

2 
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1 

1 
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23235 

8p 1 7 
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8448 
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% 
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32 

35 

33 
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[0050] Fig. 3 and Fig. 4 illustrate the adaptation to an illumination 
displaced to the red. The intensity of emission is illustrated once again 
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associated Gaussian spectral transmission curves is illustrated in the 
lower image of Fig. 3. The^SD grating optical adaptation to a red 
illumination leads via a chromatic tuning of the three grating constants 
at XI 1 1 = 728 nm RED, A.1A23 = 699 nm GREEN, M 22 = 582 nm 
BLUE to a new trichromatic RGB equilibrium position which is displaced 
to the longer wavelength end of the spectrum and forms the new white 
standard. The product of variable spectral energy distribution in the 
illuminating light and a constant triple of the Gaussian curves results in 
the new distributions of the spectral brightness values in the RGB 


diffraction orders, as they are summarized in the following table. 
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[0051] The resonant adaptation operation illustrated in Fig. 4 starts 
with a disequilibrium in the RGB diffraction orders at 43% R, 39% G, 
1 8% B, which was initiated by the sudden change from the white to 
the red illumination. Gradually progressive 3D grating optical resonance 
with longer XI 1 1 wavelengths finally leads to the chromatic grating 
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constant tuning at M 1 1 = 728 nm RED, and thus to the new RGB 


[0052] Fig. 5 and Fig. 6 show^n a similar way the 3D grating optical 
adaptation to a blue illumination via a chromatic tuning of the three 
grating constants at A.1 1 1 = 513 hm RED, M23 = 492 nm GREEN, 
A,122 = 410 nm BLUE to a new trichromatic RGB equilibrium position, 
that is displaced to the short-wave end of the spectrum and forms the 
new white standard. The product of variable spectral energy 
distribution in the illuminating light and v a constant triple of the 
Gaussian curves results in the new distributions of the spectral 
brightness values in the RGB diffraction orders, as they are summarized 
in the following ta b I e . 
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[0053] The resonant adaptation operation illustrated in Fig. 6 starts 
with a disequilibrium in the RGB diffraction orders at 24% R, 28% G, 
48% B, which was initiated by the sudden change from the white to 
the blue illumination. Gradually progressive 3D grating optical 
resonance with shorter A.1 1 1 wavelengths finally leads to the chromatic 
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grating constant tuning at A.1 1 1 =513 nm RED, and thus to the new 
RGB equilibrium at 32% R, 35% G and 33% B. 

[0054] The resonant adaptation operation also leads, of course, to a 
geometrical displacement of the position of the diffraction orders in the 
diffraction pattern, and thus with reference to the photoelectric 
receivers. However, this displacement always remains within the 
compass of the extent of the receiver surfaces. 
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